Biometric authentication system, biometric authentication method, and program

ABSTRACT

A biometric authentication system, including an image input unit configured to obtain an image by imaging a living body, a storage unit configured to store registration information relating to a plurality of biological features obtained from a biological region of an image of each person, and an authentication processing unit configured to process the biological region of the image obtained by the image input unit to execute biometric authentication based on the registration information, wherein the plurality of biological features obtained from the biological region of the each person are a plurality of biological features having a low pattern correlation with one another, and wherein the authentication processing unit is configured to combine the plurality of biological features having a low pattern correlation with one another, which are obtained by processing the image, to execute the biometric authentication.

CLAIM OF PRIORITY

The present application claims priority from Japanese patent applicationJP2018-188420, filed on Oct. 3, 2018, the content of which is herebyincorporated by reference into this application.

BACKGROUND OF THE INVENTION

This invention relates to a technology of authenticating an individualwith use of a living body.

Hitherto, as an individual authentication method for entrance/exitcontrol, attendance management, and access control, for example, controlof login to a computer, there have been widely used knowledge-basedauthentication using an identification (ID) and a password (PW) andproperty-based authentication using a physical key or an IC card, forexample. However, those authentication methods have a risk of beingforgotten or lost. As a solution thereto, biometric authentication freefrom such a risk has been recently utilized. The biometricauthentication uses an apparatus equipped with a sensor configured toread biometric information. The biometric authentication is utilized forevery access control in, for example, a personal computer (PC), a bankautomated teller machine (ATM), an entrance to a room, or a locker. Inparticular, in recent years, along with the widespread use of portableterminals such as smartphones and tablet computers, there are increasingexamples in which biometric authentication is executed on the portableterminal. Regarding the biometric authentication executed on such aportable terminal as used by everyone, authentication becomes difficultin some cases depending on biometric condition changes such as whetherwith or without glasses or a mask, a skin problem, and poor bloodcirculation. To address this, there is a demand to enable userauthentication with the use of a plurality of non-overlapping livingbody sites, for example, with the use of fingerprint authentication inplace of face authentication when a user has a mask on, and with the useof face authentication when a user has a skin trouble. In this case, itis desired to extract biometric features of a plurality of biologicaltissues at the same time without increasing the number of sensorsrequired for target living body sites, that is, with the use of a singleand general-purpose sensor, for example, a visible light camera.Further, it is important to suppress reduction in authenticationaccuracy accompanying a change in position or posture of the living bodythat occurs every time a user uses the authentication apparatus, andalso to suppress the change itself in position or posture.

In JP 2016-96987 A, there is disclosed a technology of extracting aplurality of overlapping feature amounts from an image obtained byimaging the finger, thereby executing authentication robustly againstthe change in position or posture of the living body.

In JP 2017-91186 A, there is disclosed a technology of guiding thefinger to an optimum presentation position and posture so as to suppressa change thereof, and under this state, extracting from the capturedimage a feature amount for authentication.

SUMMARY OF THE INVENTION

In order to achieve a biometric authentication apparatus with highusability and high accuracy, it is important that a plurality ofnon-overlapping biometric features can be extracted from data obtainedwith a sensor and can be used for authentication.

In JP 2016-96987 A, there is proposed a technology of extracting, froman image obtained by illuminating the finger with a light source andimaging reflected light thereof, a plurality of overlapping biometricfeatures based on information about distribution of pigmentconcentration relating to a plurality of overlapping and internalbiometric features of the finger, thereby ensuring high-accuracyauthentication. However, there is no description about a problem in thatthe plurality of biometric features cannot be easily extracted from thecaptured image depending on constraints on the light source and thesensor.

In JP 2017-91186 A, there is proposed a technology of guiding the fingerto an optimum presentation position and under this state, illuminatingthe finger with a plurality of light sources having differentwavelengths and imaging reflected light thereof to obtain an image, andthen extracting a feature amount from the image for authentication.However, there is no description about a method of guiding a pluralityof non-overlapping living body sites such as the face as well as thefinger to the optimum presentation position and imaging the living bodysites without being blocked by one another.

In view of the above-mentioned problem, this invention has an object toprovide a technology of achieving a biometric authentication apparatuswith which it is possible to guide a plurality of non-overlapping livingbody sites to an optimum presentation position and image the living bodysites without being blocked by one another, and to obtain a plurality ofbiometric feature amounts from the captured image and execute matchingof the plurality of biometric feature amounts, thereby enabling stableand highly accurate authentication.

To solve at least one of the foregoing problems, one aspect of thisinvention is a biometric authentication system, comprising: an imageinput unit configured to obtain an image by imaging a living body; astorage unit configured to store registration information relating to aplurality of biological features obtained from a biological region of animage of each person; and an authentication processing unit configuredto process the biological region of the image obtained by the imageinput unit to execute biometric authentication based on the registrationinformation, wherein the plurality of biological features obtained fromthe biological region of the each person are a plurality of biologicalfeatures having a low pattern correlation with one another, and whereinthe authentication processing unit is configured to combine theplurality of biological features having a low pattern correlation withone another, which are obtained by processing the image, to execute thebiometric authentication.

According to at least one aspect of this invention, in the biometricauthentication system, the plurality of biometric feature amounts havinga low correlation with one another are extracted from the imagesobtained through one imaging operation, and the plurality of biometricfeature amounts are subjected to matching, thereby enabling stable andhighly accurate authentication.

Problems, configurations, and effects other than the above-mentionedones become apparent from the following description of embodiments.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1A is a block diagram for illustrating an overall configuration ofa biometric authentication system for use with a finger and a faceaccording to a first embodiment of this invention.

FIG. 1B is a functional block diagram of an authentication processingunit of the first embodiment.

FIG. 2 is a diagram for illustrating an example of how to executeauthentication with the use of the biometric authentication system ofthe first embodiment.

FIG. 3 is a flowchart for illustrating processing executed at the timeof registration by the biometric authentication system of the firstembodiment.

FIG. 4 is a flowchart for illustrating processing executed at the timeof authentication by the biometric authentication system of the firstembodiment.

FIG. 5A and FIG. 5B are diagrams for illustrating an example of a guideimage displayed by the biometric authentication system of the firstembodiment.

FIG. 6 is a diagram for illustrating how to execute processing fordetecting a biological region by the biometric authentication system ofthe first embodiment with a display being used as a light source.

FIG. 7 is a flowchart for illustrating how to execute processing fordetecting a biological region by the biometric authentication system ofthe first embodiment with the display being used as a light source.

FIG. 8 is a flowchart for illustrating a detailed example of processingfor calculating a similarity with a registration data by the biometricauthentication system of the first embodiment.

FIG. 9 is a flowchart for illustrating a detailed example of processingfor calculating a similarity by the biometric authentication system ofthe first embodiment.

FIG. 10 is a flowchart for illustrating processing for extracting aplurality of feature amounts having a low correlation by the biometricauthentication system of the first embodiment.

FIG. 11 is a flowchart for illustrating processing for extracting aplurality of feature amounts having a low correlation by the biometricauthentication system of the first embodiment.

FIG. 12 is a diagram for illustrating an example of processing forextracting a feature amount by the biometric authentication system ofthe first embodiment.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

Now, referring to the accompanying drawings, description is given of atleast one embodiment of this invention. The accompanying drawings areillustrations of specific embodiments in conformity with the principleof this invention, but those are used for the understanding of thisinvention and never used to limit the interpretation of this invention.Further, components common across the respective drawings are denoted bythe same reference symbols.

First Embodiment

FIG. 1A is a block diagram for illustrating an overall configuration ofa biometric authentication system for use with a finger and a faceaccording to a first embodiment of this invention.

It should be noted that this invention may be, needless to say,configured not only as a system but also as an apparatus having all orsome of components of FIG. 1A installed in its housing. In this case,the apparatus of this invention may be an individual authenticationapparatus that encompasses authentication processing. Alternatively, theapparatus of this invention may be a biometric image obtaining apparatusdevoted to obtaining biometric images with the authentication processingbeing executed outside the apparatus.

The biometric authentication system of the first embodiment includes animage input unit 1, an authentication processing unit 2, a storage unit3, a display unit 4, and an input unit 5.

The image input unit 1 is, for example, a color camera, and isconfigured to obtain an image including biometric features from theliving body of a person to be authenticated, and input the obtainedimage to the authentication processing unit 2. The image input unit 1may be hereinafter also referred to as “camera 1.”

The authentication processing unit 2 is configured to execute imageprocessing on the image input from the image input unit 1, and executeauthentication processing. It should be noted that the image input unit1 may be also incorporated in an image processing function of theauthentication processing unit 2 to serve as an image processing unit.In any case, the authentication processing unit 2 has the imageprocessing function.

FIG. 1A is also a diagram for illustrating the configuration of theauthentication processing unit 2. The authentication processing unit 2includes a central processing unit (CPU) 6, a memory 7, and varioustypes of interfaces (IFs) 8.

The CPU 6 is configured to execute programs stored in the memory 7 toexecute various types of processing. As described below, processingexecuted by the biometric authentication system of the first embodimentwith the use of functions of an authentication module 9 or aregistration module 10 illustrated in FIG. 1B is actually executed bythe CPU 6 in accordance with the program stored in the memory 7. Thememory 7 is configured to store the programs to be executed by the CPU6. The memory 7 is also configured to temporarily store an image inputfrom the image input unit 1.

The interfaces 8 are configured to couple the authentication processingunit 2 and external apparatus. Specifically, the interfaces 8 arecoupled to, for example, the image input unit 1, the storage unit 3, thedisplay unit 4, and the input unit 5.

FIG. 1B is a functional block diagram of the authentication processingunit 2 of the first embodiment.

The authentication processing unit 2 includes the authentication module9 and the registration module 10. The authentication module 9 isconfigured to compare the data input from the image input unit 1 andregistration data registered in the storage unit 3, to authenticate auser. The registration module 10 is configured to create registrationdata from the image obtained by the image input unit 1, and store thecreated data in the storage unit 3. For example, processing of FIG. 3and processing of FIG. 4, which are to be described later, may be alsoexecuted by the registration module 10 and the authentication module 9,respectively, optionally controlling the image input unit 1, the storageunit 3, the display unit 4, and the input unit 5.

The storage unit 3 is configured to store, in advance, registration dataabout a user. The registration data is information for user matching,and is, for example, an image of a finger vein pattern. In general, theimage of the finger vein pattern is an image obtained by imaging bloodvessels (finger veins) mainly distributed beneath the finger skin on thepalm side in the form of dark shadow patterns.

The display unit 4 is, for example, a display apparatus, and serves asan output apparatus configured to display information received from theauthentication processing unit 2. The display unit 4 may be hereinafteralso referred to as “display 4.” The input unit 5 is, for example, akeyboard, and is configured to transmit information input by a user tothe authentication processing unit 2.

FIG. 2 is a diagram for illustrating an example of how to executeauthentication with the use of the biometric authentication system ofthe first embodiment.

Specifically, FIG. 2 is an illustration of how to execute authenticationwith the use of a tablet PC having the biometric authentication systeminstalled therein. In the first embodiment, the person to beauthenticated (specifically, the user) presents the living body, forexample, a face 11 and a hand 12, following a guide image and messagedisplayed on the display unit 4, for example, a display of the tabletPC. The condition of the living body is imaged by the image input unit1, for example, an in-camera of the tablet PC, and subjected toauthentication processing.

FIG. 3 is a flowchart for illustrating processing executed at the timeof registration by the biometric authentication system of the firstembodiment.

First, the system sets parameters (camera control parameter) used tocontrol the image input unit 1, such as focus, white balance, exposure,and gain, so as to obtain an image having proper quality (Step S100).Those parameters may employ a predetermined fixed value, or an optimumvalue in a past imaging operation, or under video photography, a valuedetermined by a certain rule based on an image obtained in any previousframe and a camera control parameter.

Next, the system images the living body, for example, the finger and theface with the camera to obtain an image thereof (Step S101).Subsequently, the system generates a guide image of a living body shapeas a sample for presentation and a guide message that instructs how topresent the living body in order for the person to be authenticated topresent the living body in suitable position and posture (Step S102).The guide image may indicate a part or all of the outline of the livingbody. As another example, the presented living body may be detected, andthen a rectangle or another shape consistent with or inclusive of theoutline of the detected living body may be displayed as the guide image.

Next, the system displays the guide image and message on the display(Step S103). For example, FIG. 2 is an illustration of an example ofdisplaying dotted guide images indicating the outlines of the head andthe finger, respectively, and a message “Please put your face andfingers inside the frames” in order for the person to be authenticatedto present the face and the fingers as the living body. In this case,when the guide image is displayed on an image of the living bodyobtained with the camera in a superimposed manner, the person to beauthenticated can recognize how different his or her own way to presentthe living body is from a proper way therefor, and thus can present theliving body more properly.

Next, the authentication processing unit 2 calculates an imaging qualityvalue from the captured image and the camera control parameters, forexample (Step S104). This quality value is calculated by a certain rulebased on at least one of a focus, an exposure control value, or aluminance value of an obtained image, for example. For example, aquality value of the focus may be set as a luminance contrast in aregion of the obtained image at the same position as the guide image,and a region of the obtained image from which the living body has beendetected. Further, for example, a quality value of the exposure may beset as a variation amount between an exposure control value in anyprevious frame and an exposure control value in a current frame. Stillfurther, a quality value relating to the luminance of the obtained imagemay be set as an average luminance value in the region of the obtainedimage at the same position as the guide image, or the region of theobtained image from which the living body has been detected.

Next, the authentication processing unit 2 evaluates appropriateness ofeach quality value based on a difference between the above-mentionedquality values and corresponding appropriate values, for example (StepS105). When the quality value is not appropriate, the processingproceeds to the setting of camera control parameters (Step S100). Whenthe quality value is appropriate, the authentication processing unit 2detects a biological region from the obtained image (Step S106).

The detection of the biological region may be also executed byextracting a region target for extraction of a feature amount of theface and the finger, for example, from the detected biological regionwith the use of the semantic segmentation method for classifying pixelsof the obtained image into the biological region and other regions, forexample. As another example, the object localization method forextracting a rectangular region inclusive of the biological region fromthe obtained image may be applied. As still another example, thebiological region may be extracted by any other method.

Next, the authentication processing unit 2 calculates a length, a width,an angle, and other posture information of the detected finger from theextracted biological region, for example, and then calculatesdifferences from appropriate values thereof as quality values (StepS107). Subsequently, the authentication processing unit 2 determineswhether the calculated quality value of the posture is appropriate (StepS108). When the calculated quality value of the posture is notappropriate, the processing proceeds to the setting of camera controlparameters (Step S100). When the calculated quality value of the postureis appropriate, the authentication processing unit 2 normalizes theliving body posture (Step S109).

The normalization of the posture is to clip a part of the detectedfinger region into a smaller size or extend the region throughinterpolation so as to adjust the finger length to a certain length, toenlarge or reduce the detected finger region so as to set a width of thefinger to be constant, or to rotate the finger region so as to adjustthe finger angle to a certain angle, for example.

Next, the authentication processing unit 2 extracts a feature amount formatching, from the biological region having been subjected to posturenormalization (Step S110). The extraction of a feature amount may beexecuted by extracting a fingerprint, a feature point of the face, or aline pattern of the vein, for example, from the biological region. Asanother example, a machine learning technique, for example, aconvolutional neural network (CNN), may be used to design automaticextraction of a feature amount from the biological region.

Next, the authentication processing unit 2 determines whether theextracted feature amount is appropriate (Step S111). For example, theauthentication processing unit 2 may also determine whether theextracted feature amount is a feature extracted from a real living bodyor a fake living body, for example, a picture or a print, with the useof a machine learning technique, for example, random forest or supportvector machine (SVM).

Next, the authentication processing unit 2 temporarily saves theextracted feature amount as a registration candidate (Step S112).Subsequently, the authentication processing unit 2 compares the numberof stored registration candidates with a preset number (Step S113). Whenthe number of stored registration candidates is smaller than the presetnumber, the number of stored registration candidates is insufficient,and then the processing proceeds to the setting of camera controlparameters (Step S100). When the number of stored registrationcandidates satisfies the preset number, the number of storedregistration candidates is sufficient, and then the authenticationprocessing unit 2 calculates a similarity between the feature amountsbeing registration candidates (Step S114).

Next, the authentication processing unit 2 compares the calculatedsimilarity with a preset threshold value (Step S115). When thecalculated similarity is lower than the preset threshold value, theauthentication processing unit 2 determines that the registration of theregistration candidate is to be denied, and deletes correspondingregistration candidate data stored in the memory 7. Then, the processingproceeds to the setting of camera control parameters (Step S100). Whenthe calculated similarity is higher than the threshold value, theauthentication processing unit 2 determines that the registrationcandidate is allowed to be registered (Step S116), and storescorresponding registration candidate data in the memory 7. Through theabove-mentioned processing, the registration of the biometricinformation is completed.

FIG. 4 is a flowchart for illustrating processing executed at the timeof authentication by the biometric authentication system of the firstembodiment.

In a processing flow of FIG. 4, the setting of camera control parameters(Step S100), the generation of an imaging guide (Step S101), theobtaining of a camera image (Step S102), the display of an imaging guideand a camera image (Step S103), the calculation of an imaging quality(Step S104), the appropriateness determination as to an imaging quality(Step S105), the detection of a biological region (Step S106), thecalculation of a posture quality (Step S107), the appropriatenessdetermination as to a posture quality (Step S108), the normalization ofa living body posture (Step S109), the extraction of a feature amount(Step S110), and the appropriateness determination as to a featureamount (Step S111) are the same as the processing flow for registration.

After the appropriateness determination as to a feature amount (StepS111), the authentication processing unit 2 calculates a similaritybetween the feature amount extracted in Step S110 and previouslyregistered biometric feature amount data (Step S117). Subsequently, theauthentication processing unit 2 compares the calculated similarity witha preset threshold value (Step S118). When the calculated similarity ishigher than the preset threshold value, the authentication processingunit 2 determines that the authentication is to be allowed (Step S119),and then ends the authentication processing. When the calculatedsimilarity is higher than the threshold value, the authenticationprocessing unit 2 determines that the authentication is to be denied,and the processing proceeds to the setting of camera control parameters(Step S100).

At the time of displaying the guide image and the message or the like(Step S103) of the registration and authentication processing, animaging guide, for example, images that resemble the shapes of the faceand the hand, respectively, and a message that prompts the person to beauthenticated to present the face and the hand as illustrated in FIG. 2,and the captured camera image are displayed on the display at the sametime. In a case of using a landscape display, when the guide images ofthe face and the hand are displayed side by side, it is possible toinstruct the person to be authenticated to present the face and the handside by side without difficulty. As a result, the face and the hand canbe imaged without being blocked by each other with respect to the cameraimages, and images of the face and the hand can be obtained by thesingle camera.

FIG. 5A and FIG. 5B are diagrams for illustrating an example of a guideimage displayed by the biometric authentication system of the firstembodiment.

For example, in a case of detecting regions of one or more fingers foruse in authentication from the presented hand, a guide image 13 havingan outline shape of the open hand as illustrated in FIG. 5A isdisplayed, and the hand presented in accordance with the guide image 13is imaged. With this operation, it is possible to prevent distortion ofbiological tissues such as the fingerprint, veins, and surface wrinklesof the finger caused by the contact between adjacent fingers, and thusto reduce an error in the normalization of the living body posture (StepS109) and the extraction of the feature amount (Step S110), for example.

Further, as illustrated in FIG. 5B, when the guide image 13 notincluding the outline of the base of the finger out of the outline ofthe hand is displayed, even a person to be authenticated having shorteror longer fingers than those of the hand of the guide image 13 can moreeasily fit his or her own hand to the guide image 13, leading to higherusability.

Regarding the detection of a biological region (Step S106) of theregistration and authentication processing, assuming that the display 4is a light source, the camera 1 can be said to image reflected lightthat is radiated from the light source and then reflected by the surfaceof the living body, for example, the face 11 and the hand 12. Here, ingeneral, the display 4 has a larger distance from a wall of a room beingthe background than a distance from the face 11 and hand 12 being theforeground. Considering the fact that light intensity attenuates inproportion to the square of the distance, a possible amount of lightthat is radiated from the display 4 serving as the light source,reflected by the background, and then taken by the camera 1 is muchsmaller than a possible amount of light that is reflected by theforeground and taken by the camera 1. Accordingly, in an image capturedby the camera 1 with the luminance or color of a video image to bedisplayed on the display 4 being varied, the luminance or color changesonly in the biological region being the foreground but does notsignificantly change in another region being the background. Such adifference can be used to extract the biological region.

FIG. 6 is a diagram for illustrating how to execute processing fordetecting a biological region (Step S106) by the biometricauthentication system of the first embodiment with the display 4 beingused as a light source. FIG. 7 is a flowchart for illustrating theprocessing.

In this processing, N imaging operations are carried out so as toseparate from the camera image the background being the region otherthan the biological region (Step S10601). Then, positions of obtainedimages are corrected so as to absorb camera shakes and movement of asubject (Step S10605). Then, a color difference between pixels of theobtained images subjected to the position correction is calculated (StepS10606).

At the time of N imaging operations (Step S10601), the system generatesan image to be displayed as illumination on the display 4 (Step S10602)and displays the generated image (Step S10603), and then obtains acamera image (Step S10604). Here, assuming that N=2, for example, thesystem may display the image serving as the illumination with the use ofthe display 4, for example, and under this state, capture an image of asubject with the use of a color camera 1 provided on the same surface asthe display 4. Then, the system may separate the image of the subjectinto the foreground and the background with reference to the obtainedtwo images.

The color camera 1 includes, for example, three types of light receivingelements having sensitivities to blue (B), green (G), and red (R),respectively. Those elements are arrayed in matrix in each pixel. Thelight receiving elements have spectral sensitivities that have peaks ataround 480 nanometers for blue, around 550 nanometers for green, andaround 620 nanometers for red. Through the imaging operation with thecolor camera, spatial luminance distribution of light having sensitivitypeaks at three different wavelengths can be obtained.

Further, the display 4 includes, for example, a backlight serving as alight source, a deflection filter configurated to control the luminance,and a color filter configured to control the color. The color filterincludes three types of filters that transmit blue (B) light, green (G)light, and red (R) light, respectively. Those filters are arrayed inmatrix in each pixel. The luminance is assumed to be controllable in 256levels of from 0 to 255.

In order to precisely separate the foreground and the background (StepS10607), it is required to generate an image for illumination (StepS10602) so as to maximize the color difference between the foregroundand the background in calculation of the color difference (Step S10606).For simple description, it is presumed that the product of thesensitivities of the light receiving elements for blue (B) and red (R)of the color camera is substantially 0 (specifically, one element has nosensitivity to a wavelength to which another element has a sensitivity),and a wavelength to which the RGB light receiving elements of the colorcamera have zero spectral sensitivity substantially matches a wavelengthto which the RGB color filters of the display have zero transmittance.BGR channels of an image 14 obtained through a first imaging operationare expressed by (B, G, R)=(gB1, gG1, gR1), and BGR channels of an image15 obtained through a second imaging operation are expressed by (B, G,R)=(gB2, gG2, gR2). A color difference image 16 is expressed by anumerical expression:

e=(gB1−gB2)+(gR2−gR1).

Pixels of the color difference image 16 have a difference ofsubstantially 0 at the pixels of the background not close to thedisplay, but are varied depending on the luminance or color of the imagedisplayed as the illumination on the display at the pixels of theforeground, for example, the biological region close to the display. Inthis case, when an image of (B, G, R)=(255, 0, 0) is displayed on thedisplay in the first imaging operation, and an image of (B, G, R)=(0, 0,255) is displayed on the display in the second imaging operation, theluminance in the foreground of the color difference image 16 ismaximized (first and second imaging operations can be executed in anyorder).

In this example, the authentication processing unit 2 controls thedisplay (specifically, display unit 4) to change the luminance in blue(B) and red (R), to thereby change intensities in blue (B) and red (R)of light radiated to the living body. However, this is just an example,and it is only required to control the light source so as to change anintensity in at least one color. As another example, a light sourceother than the display may be used.

The above-mentioned case is an example of a calculation method thatenables maximization of a luminance difference between the foregroundand the background in the color difference image 16 in the foregroundand background separation operation with the image on the display beingused as the illumination. When constraints on the light receivingelements of the color camera 1 and the color filters of the display 4are not satisfied and also, a different calculation method for the colordifference image 16 is to be applied, it is only required to adopt animage on the display and a calculation method for the color differenceimage that enable maximization of the luminance difference between theforeground and the background, in accordance with such conditions.

At the time of extracting a feature amount (Step S110), theauthentication processing unit 2 may extract a pattern of the vein,epidermis, or dermis of the finger or an end point or branch point ofthe fingerprint ridge as the feature amount, for example. End points ofthe eyebrow and the eye of the face and the outline of a portion aroundthe nose and the mouth may be also extracted as the feature amount, forexample. As another example, the CNN or other machine learning techniquemay be utilized to automatically design and extract a feature amountfrom the captured image.

As an example, it is assumed that the feature amount is extracted fromthe finger of the biological region with the use of the CNN. Thefingerprint of the finger skin and the vein of the finger, for example,are generally known to be uncommon to everyone. Even blood-relatedpersons have different patterns. In general, the number of variations ofthe finger existent in the real world is by far larger than a possiblenumber of variations actually prepared for training the CNN to extract afeature amount from the finger image. As a result, overfitting is liableto occur in a general CNN that executes fully connecting processingthrough alternately repeated convolution processing and poolingprocessing to connect all pixels in an image throughout layers to anoutput layer.

As one way to avoid the overfitting, there is a method using a fullyconvolutional network (FCN) being an example of the CNN, which does notadopt the fully connecting processing. Regarding the FCN, imagestructure of the input image is held up to the output image, and itslearning with a small amount of data causes less overfitting. As amethod of authenticating the person to be authenticated robustly againstthe change in position and posture with the use of a feature amountextracted by the FCN, there is known a method of gradually reducing theresolution of FCN layers with each layer to extract a low-frequencyfeature amount.

However, the above-mentioned method is not adaptable to extraction of ahigh-frequency feature amount that can be otherwise used for individualrecognition. As a method of simultaneously extracting a low-frequencyfeature amount and a high-frequency feature amount, there is known amethod of gradually reducing the resolution of the FCN layers with eachlayer and then connecting the layers in a later stage. In this case, amethod of calculating a similarity with registration data is required,which is less influenced by positional deviation while using thehigh-frequency feature amount.

FIG. 8 is a flowchart for illustrating a detailed example of processingfor calculating a similarity with the registration data (Step S117) bythe biometric authentication system of the first embodiment.

Specifically, FIG. 8 is an illustration of a flow of processing forsuppressing an influence of positional deviation while using ahigh-frequency feature amount in the calculation of a similarity withthe registration data (Step S117). First, the authentication processingunit 2 divides a registration image into M small regions (Step S1171).In a case of using a feature amount extracted from an image of thefinger, the image may be divided into three regions across the firstjoint and the second joint, for example.

Next, the authentication processing unit 2 calculates a similaritybetween an authentication image and each small region of theregistration image (Step S1172). As a measure of the similarity, theManhattan distance and the Euclidean distance can be used, for example.At the time of calculating a similarity, the authentication processingunit 2 calculates the similarity with each small region of theregistration image while scanning a certain region on the authenticationimage, and then adopts the highest similarity in the scanned region.

Next, the authentication processing unit 2 combines the M calculatedsimilarities to obtain a final similarity (Step S1173). As a combiningmethod, there may be adopted a method using an average value of the Msimilarities, for example. As described above, through adopting of thesimilarity at a position at which the highest similarity is achieved ineach small region of the registration image, it is possible to suppressan influence of the positional deviation even between feature amountsincluding high-frequency components.

Further, the following case is considered in relation to the calculationof a similarity with the registration data (Step S117): the registrationdata and the authentication are derived from a different person(matching with another person). In this case, it is preferred that thecalculated similarity be low. In a case of the registration data being aplurality of images having a low pattern correlation, for example,overlapping biological tissues, similarity in one feature at a positionat which the highest similarity is achieved in another feature amount isused for authentication, to thereby avoid such a situation that thehighest similarity with the registration data obtained through matchingwith another person is used for authentication.

This is because when the registration data and the registration data arederived from the same person, the highest similarity is achieved atalmost the same position in the above-mentioned two feature amounts, butwhen those pieces of data are derived from a different person, thehighest similarity is often achieved at different positions in theabove-mentioned two feature amounts. The above-mentioned processing issummarized in a flow of FIG. 9.

FIG. 9 is a flowchart for illustrating a detailed example of processingfor calculating a similarity by the biometric authentication system ofthe first embodiment.

At the time of calculating a similarity between the authentication imageand each small region of the registration image (Step S1172), theauthentication processing unit 2 first calculates the similarity foreach feature amount (Step S11721). For example, first, the similaritywith the registration image is calculated for a feature amount s1 (StepS11722). Next, at a position at which the highest similarity is achievedin the feature amount s1, a similarity is calculated for a featureamount s2 (Step S11723). The similarities for each feature amount arecalculated in this way. In this case, the feature amount s1 and thefeature amount s2 are different feature amounts included in S featureamounts extracted in processing to be described later with reference toFIG. 10, for example. This method is applicable to, for example, theepidermis, dermis, or vein patten of the finger.

However, the above-mentioned pattern may not be stably extracted withease as a feature amount due to the change in illumination duringimaging or the constraints on the apparatus, for example. For example,in a case of radiating near-infrared light to the finger and imagingtransmitted light thereof, a vein pattern is sharp in the obtained imageof the transmitted light, but the texture of the skin surface,specifically, the epidermis and dermis patterns, are considerablyblurry. Further, in a case of radiating visible light to the finger andimaging reflected light thereof, the epidermis and dermis pattens aresharp in the obtained image of the reflected light but the vein patternis considerably blurry. Accordingly, even when patterns of a pluralityof overlapping biological tissues are hard to extract at the same time,it is required to extract a plurality of patterns having a lowcorrelation.

FIG. 10 and FIG. 11 are flowcharts for illustrating processing forextracting a plurality of feature amounts having a low correlation bythe biometric authentication system of the first embodiment.

Specifically, FIG. 10 and FIG. 11 are each an illustration of theprocessing for extracting a plurality of feature amounts having a lowcorrelation so as to suppress the influence of positional deviation andalso to reduce the similarity in matching with another person, in thecalculation of a similarity with the registration data (Step S117). Thisprocessing is divided into processing illustrated in FIG. 10 andprocessing illustrated in FIG. 11. In the processing of FIG. 10, amachine learning model is optimized so that a feature amount extractionmodel (hereinafter referred to as “machine learning model”) generated bythe CNN or other machine learning generates a plurality of featureamounts having a low correlation from the input image. In the processingof FIG. 11, a feature amount is extracted with the use of the machinelearning model at the time of registration and authentication.

In the processing for optimizing the machine learning model asillustrated in FIG. 10, the authentication processing unit 2 first readsa plurality of images for the optimization processing (Step S120), andthen executes the detection of a biological region (Step S106), thenormalization of a living body posture (Step S109), the extraction of Sfeature amounts (Step S110), calculation of a similarity in featureamount among the read S images (Step S121), calculation of a loss valueof the machine learning model based on the calculated similarity (StepS122), and updating of parameters of the machine learning model based onthe loss value (Step S123). Finally, the authentication processing unit2 determines whether the number of times of parameter updates satisfiesa prescribed number of times (Step S124). The authentication processingunit 2 ends the processing when the prescribed number of times issatisfied, or proceeds to the reading of an image (Step S120) when theprescribed number of times is not satisfied.

In the above-mentioned processing, the detection of a biological region(Step S106) and the normalization of a living body posture (Step S109)are the same as those in the registration processing flow of FIG. 3 andthe authentication processing flow of FIG. 4. An image to be read in thereading of an image (Step S120) is basically an image obtained byimaging the living body, but may be an image of a biological regionobtained by executing the detection of a biological region (Step S106)and the normalization of a living body posture (Step S109) on thecaptured image. In this case, after the reading of an image (Step S120),the extraction of a feature amount (Step S110) may be executed directlyby the machine learning model.

The extraction of a feature amount (Step S110) is executed asillustrated in a processing flow of FIG. 11 to be described below. Thecalculation of a similarity in feature amount between images (Step S121)is the same as that in the processing flow of FIG. 8 and FIG. 9. At thetime of calculating a loss value (Step S122), when the CNN is used asthe machine learning model, a loss value, for example, contrastive lossor triplet loss, may be calculated based on the above-mentionedsimilarity, for example. At the time of updating a model parameter (StepS123), when the CNN is used as the machine learning model, an errorback-propagation method may be applied to update kernel values in eachlayer of the CNN based on the above-mentioned loss value.

In Step S122, a loss value may be also calculated so that the loss valueincreases as the similarity of a biological feature of a differentperson increases, and also so that the loss value decreases as thesimilarity of a biological feature of the same person increases. Then,in Step S123, a model parameter is updated to decrease the loss value.As a result, the feature amount extraction model for extracting, fromone biological image, a plurality of biological feature amounts having alow pattern correlation is learned.

In the extraction of S feature amounts (Step S110) as illustrated inFIG. 11, it is assumed that the CNN is used as the machine learningmodel. The authentication processing unit 2 first generates R imageshaving different resolutions from an input image (Step S1101), and thenexecutes convolution processing on the generated images (Step S1102) andcombines the thus-processed R images (Step S1103). Finally, theauthentication processing unit 2 executes convolution processing on thecombined image (Step S1104), to generate an image having S featureamounts.

At the time of generating images having different resolutions (StepS1101), the authentication processing unit 2 may, for example, reduce orenlarge the input image, or generate a low-resolution image from theinput image through convolution processing. At the time of convolutingthe R images (Step S1102), the authentication processing unit 2 mayexecute the convolution processing not once but a plurality of times,and also may execute processing for normalizing the luminance of theimage after each convolution processing. At the time of combining the Rimages (Step S1103), the authentication processing unit 2 combines imagedata thereof in a channel direction to generate image data.

At the time of final convolution (Step S1104), the authenticationprocessing unit 2 executes convolution processing so as to reduce thenumber of channels of the combined image data to S channels. Through theabove-mentioned processing, S feature amounts represented in the form ofdensity image can be obtained.

It should be noted that the processing of FIG. 10 and FIG. 11 may beexecuted by the registration module 10 or the authentication module 9 ofthe authentication processing unit 2, or alternatively executed byanother function (not shown) (for example, model optimization module).In this case, the processing of the model optimization module isexecuted by the CPU 6 in accordance with the program stored in thememory 7 as with the processing of the registration module 10 and theauthentication module 9. The processing of the model optimization moduleis executed before the registration processing of FIG. 3 and theauthentication processing of FIG. 4.

FIG. 12 is a diagram for illustrating an example of processing forextracting a feature amount by the biometric authentication system ofthe first embodiment.

Specifically, FIG. 12 is an illustration of a processing example of theabove-mentioned optimization of a machine learning model (Step S120) andthe extraction of a feature amount (Step S110) based on the optimizedmodel under the condition of S=2. A feature extraction image 18 and afeature extraction image 19 are obtained with respect to an input fingerimage 17, and those S (two in the example of FIG. 12) feature amountshave a low pattern correlation.

This is because, as a result of optimization aimed at reducing thesimilarity between feature amounts of the registration data and theauthentication data which are derived from a different person, that is,the similarity obtained through matching with another person, thefeature amounts have a lower pattern correlation, and an image positionat which the highest similarity is achieved in one feature is differentfrom an image position at which the highest similarity is achieved inanother feature.

Meanwhile, when the registration data and the authentication data arederived from the same person, that is, at the time of matching with thesame person, even when feature amounts thereof have a low patterncorrelation, those feature amounts are obtained from the same inputimage, and hence the matching positions at which the highestsimilarities with the registration data are achieved in the respectivefeature amounts are substantially the same.

As described above, the biometric authentication system according to atleast one aspect of this invention, which is illustrated in FIG. 1A andFIG. 1B, for example, includes an image input unit (for example, imageinput unit 1) configured to obtain an image by imaging a living body, astorage unit (for example, storage unit 3) configured to storeregistration information relating to a plurality of biological featuresobtained from a biological region of an image of each person, and anauthentication processing unit (for example, authentication processingunit 2) configured to process a biological region of the image obtainedby the image input unit to execute biometric authentication based on theregistration information. In this case, the plurality of biologicalfeatures obtained from the biological region of the each person are aplurality of biological features having a low pattern correlation withone another. The authentication processing unit is configured to combinethe plurality of biological features having a low pattern correlationwith one another, which are obtained by processing the image, to executethe biometric authentication.

The above-mentioned plurality of biological features having a lowpattern correlation with one another may be biological features of theface and the finger obtained by the method illustrated in FIG. 2, forexample, or S feature amounts obtained by the method illustrated in FIG.10 and FIG. 11.

With this configuration, the plurality of biological features having alow correlation with one another can be extracted from images capturedat a time. Through matching with the plurality of biological features,stable and highly accurate authentication is ensured.

Here, the living body to be imaged by the image input unit may includethe finger.

With this configuration, the biometric authentication is improved inusability.

Further, the plurality of biological features may include a firstbiological feature and a second biological feature, and theauthentication processing unit may be configured to calculate, based ona similarity between a first biological feature obtained from abiological region of the image captured by the image input unit and afirst biological feature of the registration information, a similaritybetween a second biological feature obtained from the biological regionof the image captured by the image input unit and the first biologicalfeature of the registration information.

For example, the image captured by the camera may be aligned with theimage of the registration information so as to maximize the similarityin a feature amount s 1 being one of the S feature amounts obtained bythe method illustrated in FIG. 10 and FIG. 11, for example, and then,matching with a feature amount s2 different from the feature amount s1may be performed at that position. As illustrated in FIG. 10 and FIG.11, the feature amounts s1 and s2 have a low correlation, and hence, ina case of authenticating another person, the similarity of the featureamount s2 is expected to be sufficiently low. As a result, theauthentication accuracy is improved.

Further, the image input unit may be configured to image the same livingbody a plurality of times, and the authentication processing unit may beconfigured to: control light to be radiated to the living body so as tochange an intensity of at least one color out of colors of the light tobe radiated to the living body at a time of imaging the living body theplurality of times; and extract the biological region from the imagebased on a degree of a change in intensity of the at least one color ina plurality of images obtained through imaging of the living body theplurality of times. This processing can be executed by the methodillustrated in FIG. 7, for example.

With this configuration, when the image captured by the image input unitincludes a biological region and another region, the biological regioncan be easily extracted.

Moreover, the biometric authentication system may further include adisplay unit (for example, display unit 4) configured to display animage captured by the image input unit and a guide indicating a desiredposition of the living body. In this case, the authentication processingunit may be configured to change an intensity of at least one coloroutput from the display unit so as to control the light to be radiatedto the living body. This processing can be executed by the methodillustrated in FIG. 5A and FIG. 5B, for example.

With this configuration, the posture of the living body to be imaged,for example, can be adjusted, to thereby improve the user's convenienceas well as reliably extract a biological feature. Further, with thescreen itself being used as a light source, cost for installing thesystem can be reduced.

Further, the above-mentioned plurality of biological features may beextracted from the same portion of a biological region of each image.This processing can be executed by method illustrated in FIG. 10 andFIG. 11, for example.

With this configuration, a plurality of biological features can beextracted from the images captured at a time.

In this case, the authentication processing unit may be configured to:extract the plurality of biological features from images of a pluralityof persons (for example, Step S110 of FIG. 10 and FIG. 11) with use of afeature amount extraction model; calculate a loss value of the featureamount extraction model based on a similarity among the plurality ofbiological features extracted from the images of the plurality ofpersons (for example, Step S122 of FIG. 10); and learn a parameter ofthe feature amount extraction model so as to reduce the loss value (forexample, Step S123 of FIG. 10).

With this configuration, the plurality of biological features having alow correlation with one another can be extracted from images capturedat a time.

Further, in this case, the authentication processing unit may beconfigured to: generate a plurality of images (for example, R images ofFIG. 11) having different resolutions from a biological region of theimage of the each person; generate a plurality of images throughconvolution of the plurality of images having different resolutions; andapply the feature amount extraction model to the plurality of images(for example, S images of FIG. 10 and FIG. 11) generated through theconvolution, to thereby extract a plurality of biological features.

With this configuration, a plurality of biological features having a lowcorrelation with one another can be extracted from the images capturedat a time, thereby enabling stable and highly accurate authentication.

Further, the plurality of biological features may be extracted fromdifferent portions of a biological region of each image.

Specifically, the different portions of the biological region of eachimage may be a facial portion and a finger portion of a person includedin each image as illustrated in FIG. 2 or FIG. 6, for example.

With this configuration, a plurality of biological features having a lowcorrelation with each other can be extracted from the images captured ata time, thereby enabling stable and highly accurate authentication.

This invention is not limited to the at least one embodiment describedabove, and encompasses various modification examples. For example, theat least one embodiment has described this invention in detail for theease of understanding, and this invention is not necessarily limited toa mode that includes all of the configurations described above.

Further, a part or all of the above-mentioned respective configurations,functions, processing units, processing means, and the like may beimplemented by hardware through design using, for example, an integratedcircuit. Further, the above-mentioned respective configurations,functions, and the like may be implemented by software by the processorinterpreting and executing the programs for implementing the respectivefunctions. The programs, the tables, the files, and other suchinformation for implementing the respective functions may be stored in astorage device, for example, a non-volatile semiconductor memory, a harddisk drive, or a solid state drive (SSD), or a non-transitorycomputer-readable data storage medium, for example, an IC card, an SDcard, or a DVD.

Further, the illustrated control lines and information lines are assumedto be required for the sake of description, and not all the controllines and information lines of a product are illustrated. It should beunderstood that almost all the configurations are coupled to one anotherin practical use.

1. A biometric authentication system, comprising: an image input unitconfigured to obtain an image by imaging a living body; a storage unitconfigured to store registration information relating to a plurality ofbiological features obtained from a biological region of an image ofeach person; and an authentication processing unit configured to processthe biological region of the image obtained by the image input unit toexecute biometric authentication based on the registration information,wherein the plurality of biological features obtained from thebiological region of the each person are a plurality of biologicalfeatures having a low pattern correlation with one another, and whereinthe authentication processing unit is configured to combine theplurality of biological features having a low pattern correlation withone another, which are obtained by processing the image, to execute thebiometric authentication.
 2. The biometric authentication systemaccording to claim 1, wherein the living body to be imaged by the imageinput unit includes a finger.
 3. The biometric authentication systemaccording to claim 1, wherein the plurality of biological featurescomprises a first biological feature and a second biological feature,and wherein the authentication processing unit is configured tocalculate, based on a similarity between a first biological featureobtained from a biological region of an image obtained by the imageinput unit and a first biological feature of the registrationinformation, a similarity between a second biological feature obtainedfrom the biological region of the image obtained by the image input unitand the first biological feature of the registration information.
 4. Thebiometric authentication system according to claim 1, wherein the imageinput unit is configured to image the same living body a plurality oftimes, and wherein the authentication processing unit is configured to:control light to be radiated to the living body so as to change anintensity of at least one color out of colors of the light to beradiated to the living body at a time of imaging the living body theplurality of times; and extract the biological region from the imagebased on a degree of a change in intensity of the at least one color ina plurality of images obtained through imaging of the living body theplurality of times.
 5. The biometric authentication system according toclaim 4, further comprising a display unit configured to display animage captured by the image input unit and a guide indicating a desiredposition of the living body, wherein the authentication processing unitis configured to change an intensity of at least one color output fromthe display unit so as to control the light to be radiated to the livingbody.
 6. The biometric authentication system according to claim 1,wherein the plurality of biological features are extracted from the sameportion of a biological region of each image.
 7. The biometricauthentication system according to claim 6, wherein the authenticationprocessing unit is configured to: extract the plurality of biologicalfeatures from images of a plurality of persons with use of a featureamount extraction model; calculate a loss value of the feature amountextraction model based on a similarity among the plurality of biologicalfeatures extracted from the images of the plurality of persons; andlearn a parameter of the feature amount extraction model so as to reducethe loss value.
 8. The biometric authentication system according toclaim 7, wherein the authentication processing unit is configured to:generate a plurality of images having different resolutions from abiological region of the image of the each person; generate a pluralityof images through convolution of the plurality of images havingdifferent resolutions; and apply the feature amount extraction model tothe plurality of images generated through the convolution, to therebyextract the plurality of biological features.
 9. The biometricauthentication system according to claim 1, wherein the plurality ofbiological features are extracted from different portions of abiological region of each image.
 10. The biometric authentication systemaccording to claim 9, wherein the different portions of the biologicalregion of the each image are a facial portion and a finger portion of aperson included in the each image.
 11. A biometric authenticationmethod, which is executed by a biometric authentication systemcomprising an image input unit, a storage unit, and an authenticationprocessing unit, the biometric authentication method comprising: a firststep of obtaining an image of a living body captured by the image inputunit; a second step of storing, by the authentication processing unit,registration information relating to a plurality of biological featuresobtained from a biological region of an image of each person in thestorage unit; and a third step of processing, by the authenticationprocessing unit, the biological region of the image obtained by theimage input unit to execute biometric authentication based on theregistration information, wherein the plurality of biological featuresobtained from the biological region of the image of the each person area plurality of biological features having a low pattern correlation withone another, and wherein the third step comprises combining, by theauthentication processing unit, the plurality of biological featureshaving a low pattern correlation with one another, which are obtainedthrough processing of the image, to execute the biometricauthentication.
 12. A non-transitory computer-readable storage mediumthat stores a program that control a biometric authentication system,wherein the biometric authentication system comprises an image inputunit, a storage unit, and a processor, wherein the program is configuredto cause the processor to execute: a first procedure of obtaining animage of a living body captured by the image input unit; a secondprocedure of storing, in the storage unit, registration informationrelating to a plurality of biological features obtained from abiological region of an image of each person; and a third procedure ofprocessing the biological region of the image obtained by the imageinput unit to execute biometric authentication based on the registrationinformation, wherein the plurality of biological features obtained fromthe biological region of the image of the each person are a plurality ofbiological features having a low pattern correlation with one another,and wherein the third procedure comprises a procedure of combining theplurality of biological features having a low pattern correlation withone another, which are obtained through processing of the image, toexecute the biometric authentication.